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The findings from this review do not refiect the fuii body of research evidence on Fraction Challenge. 


What is this study about? 

The study examined the effects of Fraction Challenge, 
a supplemental small-group tutoring math program 
that focuses on improving student understanding 
of fractions. 

Study authors randomly assigned fourth grade 
“at-risk” students in 53 classrooms in 13 schools 
to either an intervention group that received 
Fraction Challenge or to a comparison group 
that did not receive Fraction Challenge. At-risk 
students were identified as students who scored 
below the 35th percentile on an assessment of 
whole-number calculations. 

Students in the intervention group received tutoring 
with Fraction Challenge, an intervention that empha- 
sizes understanding of fractions using a number 
line, in groups of three students. The small groups 
met for 30 minutes at a time, three times a week, for 
12 weeks. Students in the comparison group contin- 
ued with instruction using the Floughton Mifflin Math 
program, an approach that emphasizes part-whole 
understanding of fractions, which also was used 
in intervention classrooms during non-Fraction 
Challenge time. 

The study examined the effects of Fraction Challenge 
by comparing the performance of 129 students in 
the intervention group and 130 students in the com- 
parison group on six assessments of knowledge and 
understanding of fractions that were administered 
within two weeks of the end of the intervention. 


In addition to the analysis that compared at-risk 
students in the intervention and comparison condi- 
tions, the authors also compared the at-risk stu- 
dents who received Fraction Challenge to a sample 
of “low-risk” students (defined as scoring higher 
than the 35th percentile on the assessment of 
whole-risk calculations).^ 


Features of Fraction Challenge 


Fraction Challenge is a supplemental small- 
group tutoring math program that emphasizes the 
conceptualization of fractions on a number line from 
0 to 1 (“magnitude conceptualization”), rather than 
as a part of a whole, such as % of a pie (“part-whole 
conceptualization”). The program includes scripts 
that provide models for lessons and explanations. 
Each iesson includes an Introduction of concepts, 
group practice, a speed game intended to improve 
fluency in fractions (for exampie, students wouid 
spend 1 minute identifying fractions equivalent to 
“1/2”), and individual work. Over the 12 weeks of the 
program, students are taught to: 

• Identify and name fractions 

• Understand unit fractions (fractions where the 
numerator is “1”) and the role of numerators and 
denominators 

• Place fractions on the number line 

• Compare and order fractions, and 

• Conceptualize fractions as collections of items. 
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What did the study find? 

The study authors reported, and the WWC con- 
firmed, that Fraction Challenge had statistically 
significant positive impacts on fraction knowledge. 
Students in the intervention group were better able 
to compare relative magnitudes of fractions, identify 
where fractions were on a number line from 0 to 1 , 
use magnitude and part-whole representations of 
fractions, and perform addition and subtraction 
with fractions. 


WWC Rating 


The research described in this 
report meets WWC evidence 
standards without reservations 

Strengths: The study is a well-implemented 
randomized controlled trial with low attrition. 

Cautions: The author’s comparison of the 
performance of at-risk students receiving Fraction 
Challenge to the performance of low-risk students 
did not meet WWC evidence standards, because 
the groups were not equivalent at baseline. 


August 2013 


Page 2 




WWC Single Study Review 


Appendix A: Study details 

Fuchs, L. S., Schumacher, R. F., Long, J., Namkung, J., Hamlett, C. L., Cirino, P. T., Changas, P., 
Jordan, N. C., Siegler, R., & Gersten, R. (in press). Improving at-risk iearners’ understanding 
of fractions. Journal of Educational Psychology. 


Setting 

The study was conducted in 13 schools in the United States. 

Study sample 

Fourth-grade students from 53 classrooms in 13 schools were assessed on whole-number 
calculations using the Wide Range Achievement Test-4 (WRAT-4). Students who scored 
below the 35th percentile on the WRAT-4 were Identified as “at-rlsk” students and were given 
an additional assessment: the Wechsler Abbreviated Scales of Intelligence (WASI). Eighteen 
students were excluded from the study because they scored below the 9th percentile on both 
subtests of the WASI. Students who scored above the 35th percentile on the WRAT-4 were 
labeled “low-risk” students. 

Between two and eight at-risk and low-risk students were sampled from each classroom, 
stratifying by risk severity. The at-risk students were randomly assigned to either the interven- 
tion or the comparison group, and the randomization was stratified by classroom and an indi- 
cator of risk severity (whether each student scored below the 15th percentile or between the 
15th and 34th percentiles). The analysis sample of at-risk students included 129 students in 
the intervention group and 130 students in the comparison group. The low-risk students were 
not randomized and were used for a contrast that did not meet WWC standards. 

Intervention 

group 

Students in the intervention condition received occasional small group tutoring instead of 
their regular classroom math Instruction, which used Houghton Mifflin Math. The small groups 
consisted of three students and one tutor who met for three lessons per week of 30 minutes 
each for 12 weeks. The tutors used Fraction Challenge, which contains scripts and other 
materials for the lessons. The instruction focused on conceptualizing fractions on a number 
line from 0 to 1. Each lesson included an Introduction of concepts, group practice, a speed 
game intended to improve fluency In fractions (for example, students would spend 1 minute 
identifying fractions equivalent to “1/2”), and individual work. Over the 12 weeks of the pro- 
gram, students were taught to identify and name fractions, unit fractions (fractions where the 
numerator is “1”), and the role of numerators and denominators; place fractions on the number 
line; compare and order fractions; and conceptualize fractions as collections of items. 

Comparison 

group 

Students in the comparison condition received the regular (whole class) classroom Instruction 
using Houghton Mifflin Math, which emphasizes the part-whole conceptualization of fractions. 
Part-whole conceptualization interprets fractions as representing a part of an object. Many 
of the students in the comparison group also attended a remediation course provided by the 
schools three times a week. That said, the amount of instructional time was similar for inter- 
vention and comparison students, so the effective difference in experiences across conditions 
is the use of small group tutoring (rather than whole class instruction) and the content in the 
Fraction Challenge program, relative to the use of Houghton Mifflin Math. 
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Outcomes and 
measurement 


Support for 
implementation 

Reason for 
review 


The study authors examined scores from six assessments about fractions: (a) Comparing 
Fractions, a measure from the 2010 Fraction Battery, (b) Fraction Number Line, (c) eighteen 
questions from the Nationai Assessment of Educationai Progress (NAEP) assessment during 
1990-2009 that assess part-whoie and magnitude, (d) a subscale of the 18 questions seiected 
from the NAEP assessment that address part-whole understanding, (e) a subscale of the 18 
questions selected from the NAEP assessment that address magnitude understanding, and (f) 
Fraction Calculations, a measure from the 2010 Fraction Battery. Pretests were administered 
in September and October, and posttests were administered the following April, less than 
2 weeks after the end of the intervention. For a more detailed description of these outcome 
measures, see Appendix B. 

Tutors included licensed teachers and non-licensed instructors. Tutors received a 2-day train- 
ing and met bi-weekly to discuss tutoring topics and challenges. Each tutor worked with two 
to four small groups. 

This study was identified for review by the WWC because it was supported by a grant to the 
University of Delaware (Principal Investigator: Nancy Jordan) from the National Center for 
Special Education Research (NCSER) at the Institute of Education Sciences (lES). 
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Appendix B: Outcome measures for the mathematics achievement domain 


Mathematics achievement 

Comparing Fractions 

This measure is from the 2010 Fraction Battery and consists of 15 pairs of fractions. Students write an inequai- 
ity sign to reiate the fractions in each pair. Each correctiy-answered item is worth one point. Internai consistency 
for this assessment was 0.84. 

Eighteen items seiected from NAEP 
during 1990-09 (“Items from NAEP”) 

The researchers seiected 18 items about fractions from the NAEP during 1990-2009. Eight items assess part- 
whoie understanding, eight items assess magnitude understanding, one item requires subtraction of two frac- 
tions with the same denominator, and one asks how many quarters there are in a whoie. internai consistency for 
this assessment was 0.72. 

Magnitude subscale of the Items 
from NAEP 

This measure consists of eight items in the items from NAEP outcome that assess magnitude understanding 
of fractions. Magnitude conceptuaiization interprets fractions as points on a number iine from 0 to 1. internai 
consistency for this assessment was 0.62. 

Part- whole subscale of the Items 
from NAEP 

This measure consists of eight items in the items from NAEP outcome that assess part-whoie understanding 
of fractions. Part-whoie conceptuaiization interprets fractions as representing a part of an object, internai 
consistency for this assessment was 0.60. 

Erection Calculations 

This measure is from the 2010 Eraction Battery and consists of 10 fraction addition questions and 10 fraction 
subtraction questions, internai consistency for this assessment was 0.90. 

Eraction Number Line 

This measure gives 10 fractions to a student and asks him or her to identify the corresponding point on a 
number iine from 0 to 1. The score is the sum of the absoiute differences between the fraction and the student’s 
piacement on the iine. Test-retest reiiabiiity for this assessment was 0.79. 
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Appendix C: Study findings for the mathematics achievement domain 





Mean 

(standard deviation) 


WWC calculations 


Domain and 
outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Mathematics achievement 

Comparing Fractions 

At-risk 

students 

13 schools/ 
259 students 

12.91 

(3.37) 

7.07 

(2.84) 

5.84 

1.87 

+A7 

0.00 

Fraction Calcuiations 

At-risk 

students 

13 schools/ 
259 students 

17.57 

(3.76) 

7.50 

(4.30) 

10.07 

2.49 

+A9 

0.00 

Fraction Number Line 

At-risk 

students 

13 schools/ 
259 students 

-0.21 

(0.09) 

-0.32 

(0.12) 

0.11 

1.03 

+2,5 

0.00 

items from NAEP 

At-risk 

students 

13 schools/ 
259 students 

14.36 

(3.11) 

11.35 

(3.43) 

3.01 

0.92 

+22 

0.00 

Domain average for mathematics achievement 




1.58 

+44 

Statistically 

significant 


Table Notes: Positive resuits for mean difference, effect size, and improvement index favor the intervention group; negative results favor the comparison group. The signs of 
the means for the Fraction Number Line outcome were made negative so that a positive mean difference would reflect a favorable impact of the intervention. The effect size is a 
standardized measure of the effect of an intervention on student outcomes, representing the change (measured in standard deviations) in an average student's outcome that can 
be expected if the student is given the intervention. The improvement index is an alternate presentation of the effect size, reflecting the change in an average student’s percen- 
tile rank that can be expected if the student is given the intervention. The WWC-computed average effect size is a simple average rounded to two decimal places; the average 
improvement index is calculated from the average effect size. The statistical significance of the study’s domain average was determined by the WWC. NAEP = National Assess- 
ment of Educational Progress. 

Study Notes: A correction for multiple comparisons was needed but did not affect significance levels. The p-values presented here were calculated by the WWC. The WWC 
calculated the intervention group mean by adding the difference-in-differences adjusted estimate of the average impact of the program (i.e., difference in mean gains between the 
intervention and comparison groups) to the unadjusted comparison group posttests means. Please see the WWC Handbook v2.1 for more information. The average impacts of the 
program are calculated from the means reported in the study instead of the estimation model reported in the study because the model included an interaction term of the interven- 
tion with the pretest score for each outcome measure, and therefore, the effect of the intervention would differ depending on the score on the pretest. 

The study is characterized as having a statistically significant positive effect because univariate statistical tests are reported for each outcome measure, the effect for at least one 
measure within the domain is positive and statistically significant, and no effects are negative and statistically significant, accounting for multiple comparisons. 
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Appendix D: Supplemental findings by domain 


Domain and 
outcome measure 

Study 

sample 


Mean 

(standard deviation) 

WWC calculations 


Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Mathematics achievement 

Magnitude subscale of the 

At-risk 

13 schools/ 

6.96 

4.66 

2.30 

1,07 

-h36 

0,00 

Items from NAEP 

students 

259 students 

(2.21) 

(2,08) 





Part-whole subscale of the 

At-risk 

13 schools/ 

5.78 

5,36 

0,42 

0.29 

-hII 

0.02 

Items from NAEP 

students 

259 students 

(1.12) 

(1.71) 






Table Notes: Positive results for mean difference, effecf size, and improvement index favor the intervention group; negative results favor the comparison group. The effect size is 
a standardized measure of fhe effect of an intervenfion on studenf outcomes, representing the change (measured in standard deviations) in an average student’s outcome that can 
be expected if fhe sfudenf is given fhe intervention. The improvement index is an alternate presentation of the effect size, reflecting the change in an average student’s percentile 
rank that can be expected if fhe studenf is given fhe intervention. 

Study Notes: A correction for clusfering was needed buf did not affect significance levels. The p-values presenfed here were calculafed by the WWC. The WWC calculated the 
intervention group mean by adding the difference-in-differences adjusted estimate of fhe average impact of the program (i.e., difference in mean gains between the intervention 
and comparison groups) to the unadjusted comparison group posttests means. Please see the WWC Handbook v2.1 for more information. The average impacts of the program 
are calculated from the means reported in the study instead of the estimation model reported in the study because the model included an interaction term of the intervention with 
the pretest score for each outcome measure, and therefore, the effect of fhe infervention would differ depending on fhe score on fhe pretesf. These findings are nof included in 
Appendix C because fhe two subscales make up the total score for fhe “Items from NAEP’’ oufcome. 
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Endnotes 

■' Single study reviews examine evidence published in a study (supplemented, if necessary, by information obtained directly from the 
author[s]) to assess whether the study design meets WWC evidence standards. The review reports the WWC’s assessment of whether 
the study meets WWC evidence standards and summarizes the study findings following WWC conventions for reporting evidence on 
effectiveness. This study was reviewed using the Elementary School Math review protocol, version 2.0. The WWC rating applies only 
to the results that were eligible under this topic area and met WWC standards without reservations or met WWC standards with reser- 
vations, and not necessarily to all results presented in the study. 

^ Because the two groups of students in this contrast were not randomly assigned to receive Fraction Challenge, the WWC consid- 
ers this analysis to be based on a quasi-experimental design that must demonstrate baseline equivalence to meet WWC standards 
with reservations. The intervention and comparison groups in this contrast were not equivalent at baseline, so this portion of the study 
does not meet WWC standards. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, August). 

WWC review of the report: Improving at-risk learners’ understanding of fractions. Retrieved from 
http://whatworks.ed.gov 
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Glossary of Terms 

Attrition 


Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Improvement index 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Single-case design 
(SCD) 

Standard deviation 


Statistical significance 
Substantively important 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review if it falls within the scope of the review protocol and uses either 
an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

Along a percentile distribution of students, the improvement index represents the gain 
or loss of the average student due to the intervention. As the average student starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned 
to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into intervention and comparison groups. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 2.1) for additional details. 


August 2013 


Page 9 


